Chapter 28 



Noise 



He who establishes his argument by noise and command 
shows that his reason is weak. 
— M. de Montaigne 



This chapter (which reader can safely skip on the first reading) is about noise, 
how it affects classical dynamics, and the ways it mimics quantum dynam- 
ics. 

Why - in a monograph on deterministic and quantum chaos - start discussing 
noise? First, in physical settings any dynamics takes place against a noisy back- 
ground, and whatever prediction we might have, we have to check its robustness to 
noise. Second, as we show in this chapter, to the leading order in noise strength, 
the semiclassical Hamilton- Jacobi formalism applies to weakly stochastic flows 
in toto. As classical noisy dynamics is more intuitive than quantum dynamics, 
understanding effects of noise helps demystify some of the formal machinery of 
semiclassical quantization. Surprisingly, symplectic structure emerges here not 
as a deep principle of mechanics, but an artifact of the leading approximation to 
quantum/noisy dynamics. Third, the variational principle derived here turns out 
to be a powerful tool for determining periodic orbits, see chapter 29. And, last but 
not least, upon some reflection, the whole enterprize of replacing deterministic 
trajectories by deterministic evolution operators, chapters 16 to 20, seems fatally 
flowed; if we have given up infinite precision in specifying initial conditions, why 
do we alow ourselves the infinite precision in the specification of evolution laws, 
i.e., define the evolution operator by means of the Dirac delta function 5{y-f\x))l 
It will be comforting to learn that the deterministic evolution operators survive un- 
scathed, as the leading approximation to the noisy ones in the limit of weak noise. 

Another key result derived here is the evolution law (28.45) for the covariance 
matrix Q a of a linearly evolved Gaussian density, 

Qa+\ = M a Q a M a T + ^ . 
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To keep things simple we shall describe covariance evolution in the discrete time 
dynamics context, but the results apply both to the continuous and discrete time 
flows. The most important lesson, however, is that physicist's Brownian diffusion 
intuition -that the effect of the noise is to spread out the trajectory as is wrong: 
In nonlinear dynamics the noise is always local, determined by balancing local 
nonlinear dynamics against the memory of the noise past. section 28.5 

We start by deriving the continuity equation for purely deterministic, noiseless 
flow, and then incorporate noise in stages: diffusion equation, Langevin equation, 
Fokker-Planck equation, stochastic path integrals, Hamilton- Jacobi formulation. 



The large body of accrued wisdom on the subject of flows called fluid dynamics 
is about physical flows of media with continuous densities. On the other hand, the 
flows in state spaces of dynamical systems frequently require more abstract tools. 
To sharpen our intuition about those, it is helpful to outline the more tangible fluid 
dynamical vision. 

Consider first the simplest property of a fluid flow called material invariant. 
A material invariant I(x) is a property attached to each point x that is preserved 
by the flow, I(x) = /(/ f (jc)); for example, at point x{t) = f\x)) a green particle 
(more formally: a passive scalar) is embedded into the fluid. As I(x) is invariant, 
its total time derivative vanishes, t{x) = 0. Written in terms of partial derivatives 
this is the conservation equation for the material invariant 



Let the density of representative points be p(x, t). The manner in which the flow 
redistributes I(x) is governed by a partial differential equation whose form is rel- 
atively simple because the representative points are neither created nor destroyed. 
This conservation property is expressed in the integral statement 



where V is an arbitrary volume in the state space M, dV is its surface, h is its out- 
ward normal, and repeated indices are summed over throughout. The divergence 
theorem turns the surface integral into a volume integral, 



28.1 Deterministic transport 



(E.A. Spiegel and P. Cvitanovic) 



d,I + v • di = . 



(28.1) 



d t I dxpl - - 
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where 3, is the partial derivative operator with respect to Xj. Since the integration 
is over an arbitrary volume, we conclude that 

d t (pl) + diiplvi) = . (28.2) 

The choice 1=1 yields the continuity equation for the density: 

d t p + di(p Vi ) = . (28.3) 

Here we have used the language of fluid mechanics to ease the visualization, but, 
as we have seen in our previous derivation of the continuity equation (16.25), any 
deterministic state space flow satisfies the continuity equation in any dimension. 

Why -even though the dynamics is nonlinear- is this equation linear? As each 
deterministic orbit is distinct and intersects no other orbit, no 'particles' are cre- 
ated or destroyed, they are non-interacting, hence description in terms of linear 
evolution operators possible. 



28.2 Brownian diffusion 



Consider tracer molecules, let us say big, laggardly green molecules, embedded 
in a denser gas of light molecules. Assume that the density of tracer molecules p 
compared to the background gas density is low, so we can neglect green-green col- 
lisions. Each green molecule, jostled by frequent collisions with the background 
gas, executes its own Brownian motion. The molecules are neither created nor 
destroyed, so their number within an arbitrary volume V changes with time only 
by the current density y, flow through its surface dV (with h its outward normal): 

d t I dxp-- I dcrhijj. (28.4) 
Jv JdV 

The divergence theorem turns this into the conservation law for tracer density: 

d t p + diji = 0. (28.5) 

The tracer density p is defined as the average density of a 'material particle,' av- 
eraged over a subvolume large enough to contain many green (and still many 
more background) molecules, but small compared to the macroscopic observa- 
tional scales. What is p. If the density is constant, on the average as many 
molecules leave the material particle volume as they enter it, so a reasonable phe- 
nomenological assumption is that the average current density {not the individual 
particle current density pv,- in (28.3)) is driven by the density gradient 

ji = -D^- . (28.6) 
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This is the Fick law, with the diffusion constant D a phenomenological parameter. 
Substituting this current into (28.5) yields the diffusion or heat equation, 

-p{x,t) = D—p{x,t) . (28.7) 

More generally, diffusion is described by a space- and time-dependent symmet- 
ric diffusion tensor A,y - A y; -, with j; = -jAjjdjp, leading to the anisotropic 
diffusion equation 

d t p(x, t) = -di (Aij(x) d jP (x, 0) . (28.8) 

For sake of streamlining the argument we have assumed above that diffusion in d 
dimensions is homogenous and iso tro pic, A(x) = 2 D 1. In practice, the diffusion 
tensor is almost always anisotropic: for example, physicist's Brownian diffusion 
is a flow in the 6-dimensional {configuration, velocity} phase space, with white 
noise probability distribution exp(-v 2 /2ksT), modeling random force kicks ap- 
plied only to the 3 velocity variables v. In this case one thinks of diffusion coeffi- 
cient D = ksT/2 as temperature. 



28.2.1 Heat kernel 



Fourier transforming the heat equation (28.7), 



d_ 

df 



/dk 
—p(k,t)e lkx , (28.9) 
2n 



integrating, 



Cdk 



PU.I)= I ^k,0)e""- Dk ''. 



and Fourier transforming back we obtain an exact solution of the heat equation in 
terms of an initial Dirac delta density distribution, p(x, 0) = 6(x - xq), 

p(x,t) = £ FP (x,t;x ,0) = !— — e~^w" t (28.10) 

{AnDt) 1 

in the spirit of the quantum free particle propagation of sect. 33.2.2. The average 
distance covered in time t obeys the diffusion formula 

((x-x ) 2 } t = J dx p(x, t)(x - x ) 2 = 2dDt . (28.11) 

The classical Einstein formula describes 3-dimensional Brownian motion; here 
the diffusion takes place in the dynamical state space of dimension d. 
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28.2.2 Random walks 

So far we have considered the evolution of the density of tracer molecules. One 
can alternatively consider a J-dimensional random walk of an individual tracer 
molecule kicked by a stochastic term, 

§ = fft). (28.12) 

A way to make sense of |(f) is to first construct the probability distribution for 
additive noise ^ at short but finite time steps St, with t n+ \ = t n + St, and the 
particle x n = x(t n ) at time t n executing a random walk, x n+ \ - x„ + g(t n ) , where x 
is a ^-dimensional state vector, and x n j is its jth component at time n. The natural 
choice is that probability that the trajectory reaches x n+ \ is given by a normalized 
Gaussian 



■£>Fp(Xn+l> tn+l i tn) — ) _ __ _ exp 



1 

V(27r5T) rf detA 



1 T J_ 



, (28.13) 



% n - x n+ \ - x n , characterized by zero mean and the diffusion tensor (covariance 
matrix), 

(£;(*„)) = , {ti(t m )Zj T (t n )) - 6tA u 5 nm , (28. 14) 

where (• • •) stands for ensemble average over many realizations of the noise, and 
the superfix T indicates a transpose. As the time discretization 5t is arbitrary, 
the diffusing cloud of noisy trajectories should be described by a distribution that 
keeps its form as 5t —> 0. Indeed, the semigroup property of a Gaussian kernel, 

£, FP {x,t;x",t") = J"dx'£, FP (x, t; x , t') £, FP (x , t'; x" , t") , (28.15) 

ensures that the distribution keeps its form under successive diffusive steps. 
£, FP {x, t; xo, 0) describes the diffusion at any time, including the integer time in- 
crements {?„} = {St, 26t, ■ ■ ■ ,uSt, ■ ■ ■}, and thus provides a bridge between the 
continuous and discrete time formulations of noisy evolution. 



Example 28.1 Random walk in one dimension The white noise g„ = x„ +l - x n 
for a 1 -dimensional diffusion process is a normally distributed random variable, with 
standard normal (i.e., Gaussian) probability distribution function, 



£ FP (x, t; x, t') 1 exp 



(x-x') 



'\2 



(28.16) 



AD(t-t') 

of mean 0, variance 2D(t-t'), and standard deviation yj2D(t - f), uncorrelated in time: 

<x„ + i - x n ) = , <(x„ !+ i - x m )(x„ +l - x n )} = 2D5t5,„„ . (28.17) 

section 28.4 
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28.3 Noisy trajectories: Continuous time 



The connection between path integration and Brownian 
motion is so close that they are nearly indistinguishable. 
Unfortunately though, like a body and its mirror image, 
the sum over paths for Brownian motion is a theory hav- 
ing substance, while its path integral image exists mainly 
in the eye of the beholder. 
— L. S. Schulman 

(P. Cvitanovic and D. Lippolis) 

So far we have considered tracer molecule dynamics which is purely Brownian, 
with no deterministic "drift." Consider next a J-dimensional deterministic flow 
x = v(x) perturbed by a stochastic term %(f), 



dx 

— = v(x) + 
at 



(28.18) 



where the deterministic velocity field v{x) is called 'drift' in the stochastic litera- 
ture, and f (t) is additive noise, uncorrected in time. We shall refer to equations 
of this type as Langevin equations. The more general case of a tensor A(x) which 
is a state space position dependent but time independent can be treated along the 
same lines. In this case the stochastic flow (28.18) is written as 



dx = v(x) dt + o-(x) d£(t) , 



15„ 



o~ o~ 



(28.19) 



cr(jc) is called the 'diffusion matrix', and the noise is referred to as 'multiplicative'. 
Explicit time dependence in A(x, t) would take us into world of non-autonomous, 
externally driven flows, beyond the comfort zone of ChaosBook.org. 

As in (28.12), a way to make sense of (28.18) is to first construct the proba- 
bility distribution for additive noise £ at a short but finite time 8t. In time St the 
deterministic trajectory advances by v(x n ) St. As 6t is arbitrary, it is desirable that 
the diffusing cloud of noisy trajectories is given by a distribution that keeps its 
form as St — > 0. This holds if the noise is Brownian, i.e., the probability that the 
trajectory reaches x„ + i is given by a normalized Gaussian (28.13), 



£ FP (x„+i,6Tix n ,0) = -^exp 



1 T J_ 

2 St A„ 



(28.20) 



Here = 6x„ - v(x n )6r, the deviation of the noisy trajectory from the deter- 
ministic one, can be viewed either in terms of velocities {x, v(x)} (continuous time 
formulation), or finite time maps \x n — > x, 1+i ,x n — > f ST {x n )} (discrete time formu- 
lation), 



6x„ = x„+i - x n - x„ 5t , f T (x n ) -x n - v(x n ) 5t , 



(28.21) 
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where 



{xq, x\, • • • , x n , • ■ • , Xk) = {x(0), x(St), ■ ■ ■ , x(n6r), ■ ■ ■ , x(t)} 



(28.22) 



is a sequence of k + 1 points x n = x(t n ) along the noisy trajectory, separated by 
time increments 5t - t/k. 

The phenomenological Fick law current (28.6) is now a sum of two compo- 
nents, the material particle deterministic drift v(x) and the weak noise term 



ji = Vjp - D— , 

OX; 



vtp- -Aij(x)djp(x,f) 



(28.23) 



with the full, anisotropic and space-dependent version indicated in [• • •]. Substi- 
tuting this j into (28.5) yields the Fokker-Planck equation 



d t p + diipvi) = Dd 2 p, 



- ^d i {A ij (x)d j p(x,t)) 



(28.24) 



The left hand side, dp/dt = d t p + d ■ (pv), is deterministic, with the continuity 
equation (28.3) recovered in the weak noise limit D — > 0. The right hand side 
describes the diffusive transport in or out of the material particle volume. If the 
density is lower than in the immediate neighborhood, the local curvature is posi- 
tive, d p > 0, and the density grows. Conversely, for negative curvature diffusion 
lowers the local density, thus smoothing the variability of p. Where is the density 
going globally? 

If the system is bound, the probability density vanishes sufficiently fast outside 
the central region, p(x, t) — > as \x\ — > oo, and the total probability is conserved 



dxp(x, t) = 1 . 



Any initial density p(x, 0) is smoothed by diffusion and with time tends to the 
natural measure, the invariant density 



po(x) - lim p(x, t) , 

t—>oo 



(28.25) 



an eigenfunction p(x, t) = e st po{x) of the time-independent Fokker-Planck equa- 
tion 



(djVt-Dd 2 + s a )p a = 0, 



(28.26) 



with vanishing eigenvalue so = 0. Provided the noiseless classical flow is hyper- 
bolic, in the vanishing noise limit the leading eigenfunction of the Fokker-Planck 
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equation tends to natural measure (16.17) of the corresponding deterministic flow, 
the leading eigenvector of the Perron-Frobenius operator. 

If the system is open, there is a continuous outflow of probability from the 
region under study, the leading eigenvalue is contracting, so < 0, and the density 
of the system tends to zero. In this case the leading eigenvalue so of the time- 
independent Fokker-Planck equation (28.26) can be interpreted by saying that a 
finite density can be maintained by pumping back probability into the system at 
a constant rate y = -sq. The value of y for which any initial probability density 
converges to a finite stationary equilibrium density is called the escape rate. In 
the noiseless limit this coincides with the deterministic escape rate (17.36). 

The distribution (28.13) describes how an initial density of particles concen- 
trated in a Dirac delta function at x n spreads in time 5t. In the Fokker-Planck 
description individual noisy Langevin trajectories (28.18) are replaced by the evo- 
lution of the density of noisy trajectories. The finite time Fokker-Planck evolution 
p(x, t) = [£,' F p o p] (x, 0) of an initial density p(xo, 0) is obtained by a sequence of 
consecutive short-time steps (28.13) 

£ FP (x k , t; xq, 0) = J [dx] exp Z [x " +1 ~ /^M 2 } - < 28 - 27 ) 

where t = U8t, and the Gaussian normalization factor in (28.13) is absorbed into 
intermediate integrations by denning 

m - 0^ 

N„ = (27i5T) d/2 [det A(x n )] 1/2 (anisotropic diffusion tensor A) 

= (4DdT) d/2 (isotropic diffusion, A(jc) = 2D 1) . (28.28) 

As D —> 0, the distribution tends to the noiseless, deterministic Dirac delta func- 
tion Perron-Frobenius operator (16.10). The stochastic flow (28.18) can now be 
understood as the continuous time, St — > limit, with the velocity noise %{t) a 
Gaussian random variable of zero mean and covariance matrix 

fait)) = , (|,(0 Ijit'j) = A, 7 6(t - O . (28.29) 

It is worth noting that the continuous time flow noise f(£) in (28.18) and (28.29) is 
dimensionally a velocity M/M, as £ FP (x n+ \, 5t; x n , 0) is a probability density for 
velocity while the discrete time noise in (28.13), (28.14) is dimensionally a 
length [x], as p(x, t) is a probability density for position x. The important point is 
that the same diffusion tensor A(x) describes the diffusion both in the configuration 
space and the velocity space. 
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The continuous time limit of (28.27), 8t = t/k —> 0, defines formally the 
Fokker-Planck evolution operator 

£ FP (x, t; xq, 0) - J [dx] exp [x(t) - v(x(r))] 2 Jr| (28.30) 



as a stochastic path (or Wiener) integral for a noisy flow, and the associated con- 
tinuous time Fokker-Planck (or forward Kolmogorov) equation (28.24) describes 
the time evolution of a density of noisy trajectories. We have introduced noise 
phenomenologically, and used the weak noise assumption in retaining only the 
first derivative of p in formulating the Fick law (28.6) and including noise addi- 
tively in (28.23). The 8t — > limit and the proper definition of x(t) are delicate 
issues of no import for the applications studied here. A full theory of stochastic 
ODEs is much subtler, but this will do for our purposes. 

The exponent 

- ThTT h+i - /' r (*«)f * - TFT W r ) " V « T ))] 2 ( 28 - 31 ) 
4D ot L J 4D 

can be interpreted as a cost function which penalizes deviation of the noisy trajec- 
tory Sx from its deterministic prediction v 5t, or, in the continuous time limit, the 
deviation of the noisy trajectory tangent x from the deterministic velocity field v. 
Its minimization is one of the most important tools of the optimal control theory, 
with velocity x(t) along a trial path varied with aim of minimizing its distance to 
the target v(jc(t)). 



28.4 Noisy maps: Discrete time 



(P. Cvitanovic and D. Lippolis) 



For pedagogical reasons we shall often find it convenient to consider a noisy dis- 
crete time dynamical system 

Xn+l =/(*»)+ (28.32) 

where x is a <i-dimensional state vector, and x n j is its jth component at time n. 
In the Fokker-Planck description individual noisy trajectories are replaced by the 
evolution of the density of noisy trajectories, with the g n = x n +\ -f(x n ) probability 
distribution of zero mean and diffusion tensor, and the time increment in (28. 14) 
set to 6t - 1 , 

{U}) = , (%n,i fmj) = ^Mn) $nm ■ (28.33) 
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As we shall show, in nonlinear dynamics the noise is never isotropic and/or ho- 
mogeneous. Even if the infinitesimal time step noise (28.13) covariance matrix 
in (28.19) were independent of the state space position x, this cannot be true of 
A(x) for the discrete time flow (28.32) obtained by the Poincare section reduction 
method of sect. 3.1, as the return times (3.1) and the noise accumulated along the 
corresponding trajectory segments depend on the starting Poincare section point. 
Indeed, as we shall argue in sect. 28.5, in nonlinear dynamics all noise is local. 
As long as the noise distribution at x is autonomous (not explicitly dependent on 
time) the stochastic flow (28.32) can be written as x n+ \ = x n + cr{x n )^ n , where 
A = cro" T , and o~{x) is the multiplicative noise diffusion matrix defined in (28.19). 



The action of discrete one-time step Fokker-Planck evolution operator on the 
density distribution p at time k, 



Pk+\(y) = [£ FP pk](y) = J dx£ FP ( 

*fpM = -^-e-^- m)T W>^ x) \ (28.34) 
N{x) 



is centered on the deterministic step f(x) and smeared out diffusively by noise. 
Were diffusion uniform and isotropic, A(x) = 2D I, the Fokker-Planck evolution 
operator would be proportional to exp [-{y - f(x)} 2 /2A^j, i.e., the penalty for stray- 
ing from the deterministic path is just a quadratic error function. The kth iterate 
of £ k FP {xk\ xq) - £ FP (xk, f, xq, 0) is a J-dimensional path integral over the k - 1 
intermediate noisy trajectory points, 

£ k FP (x k ;x Q ) = J [j x ]^3 2«fe + i-/fe) T )A(bfe + i-/^)) ? (28.35) 

where the Gaussian normalization factor in (28.34) is absorbed into intermediate 
integrations by defining 

[dx] = Y\ -jf , N n = > /(27r) d det A(x n ) . (28.36) 

n—\ 

We shall also need to determine the effect of noise accumulated along the 
trajectory points preceding x. As the noise is additive forward in time, one cannot 
simply invert the Fokker-Planck evolution operator; instead, the past is described 
by the adjoint Fokker-Planck evolution operator, 

Pk-i(x) = L4pP*](*)= fldy] e-^-f^^-f^pkiy), (28.37) 

which transports a density concentrated around the point f(x) to a density con- 
centrated around the previous point x and adds noise to it. In the deterministic, 
vanishing noise limit this is the Koopman operator (G.l). 
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The Fokker-Planck evolution operator (28.34) is non-hermitian and non-unitary. 
For example, if the deterministic flow is contracting, the natural measure (the lead- 
ing right eigenvector of the evolution operator) will be concentrated and peaked, 
but then the corresponding left eigenvector has to be broad and flat, as backward 
in time the deterministic flow is expanding. We shall denote by p a the right eigen- 
vectors of -L FP , and by p a its left eigenvectors, i.e., the right eigenvectors of the 
adjoint operator £ FP . 



Our main goal in this section is to convince the reader that the diffusive dynamics 
of nonlinear flows is fundamentally different from Brownian motion, with the flow 
inducing a local, history dependent noise. In order to accomplish this, we gener- 
alize here the notion of invariant deterministic recurrent solutions, such as fixed 
points and periodic orbits, to noisy flows. While a Langevin trajectory (28.32) 
can never return exactly to the initial point and thus cannot ever be periodic, in 
the Fokker-Planck formulation (28.35) a recurrent motion can be defined as one 
where a peaked distribution returns to the initial neighborhood after time n. Re- 
currence so defined not only coincides with the classical notion of a recurrent orbit 
in the vanishing noise limit, but it also enables us to derive exact formulas for how 
this local, history dependent noise is to be computed. 

As the function x n +\ - f{x n ) is a nonlinear function, in general the path inte- 
gral (28.35) can only be evaluated numerically. In the vanishing noise limit the 
Gaussian kernel sharpens into the Dirac ^-function, and the Fokker-Planck evo- 
lution operator reduces to the deterministic Perron-Frobenius operator (16.10). 
For weak noise the Fokker-Planck evolution operator can be evaluated perturba- 
tively as an asymptotic series in powers of the diffusion constant, centered on the 
deterministic trajectory. Here we retain only the linear term in this series, which 
has a particulary simple dynamics given by a covariance matrix evolution formula 
(see (28.45) below) that we now derive. 

We shift local coordinates labeled at time 'a' to the deterministic trajectory 
{. . . , x-i, xq, xi, X2, . . . , } centered coordinate frame x = x a + z a , Taylor expand 

f(x) = fa{Za) - x a +i + M a z a H , and approximate the noisy map (28.32) by its 

linearization, 



28.5 All nonlinear noise is local 



I ain't gonna work for Maggie's pa no more 
No, I ain't gonna work for Maggie's pa no more 
Well, he puts his cigar 
Out in your face just for kicks 



— Bob Dylan, Maggie 's Farm 



(P. Cvitanovic and D. Lippolis) 



Za+l = M a Z a + £ 



a i 



Mij{x) = dfi/dxj 



(28.38) 
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with the deterministic trajectory points at z a - Z a +\ - 0, and M a - M(x a ) the 
one time step Jacobian matrix. The corresponding linearized Fokker-Planck evo- 
lution operator (28.34) action on density p a {z a ) = p(x a +z a ,a) is given in the local 
coordinates by 

J dZa£ a FP {Za + l,Za)Pa{Za) (28.39) 

by the linearization (28.38) centered on the deterministic trajectory 

■L a FP {Za+\,Za) = L e -l^-^z a ) T i- a (z a+i -M a z a ) _ (26.40) 

The superscript 'a' in £ a Fp distinguishes the local, linearized Fokker-Planck evo- 
lution operator coordinate frame z a = x- x a centered on the deterministic trajec- 
tory point x a from the full global evolution operator (28.35), in global coordinate 
system x. 

The kernel of the linearized Fokker-Planck evolution operator (28.40) is a 
Gaussian. As a convolution of a Gaussian with a Gaussian is again a Gaussian, 
we investigate the action of the linearized Fokker-Planck evolution operator on a 
normalized, cigar-shaped Gaussian density distribution 



Pa(z) = -J-e^E * , C a = (2^ /2 (det Q a ) l/2 , (28.41) 
(-a 



and the action of the linearized adjoint Fokker-Planck evolution operator on den- 
sity 

Paiz) = ^e'^-t z , C a = (2/r) d/2 (det Q a ) l/2 , (28.42) 

also centered on the deterministic trajectory, with strictly positive [dxd] covari- 
ance matrices Q, Q. Label V plays a double role, and {a + I, a} stands both for 
the {next, initial) space partition and for the times the trajectory lands in these 
partitions. The linearized Fokker-Planck evolution operator (28.40) maps the 
Gaussian p a (z a ) into the Gaussian 

p a+ i(Za+i) = — f [dz a ]e~^ iZa+1 ~ MaZa)T '^ (Za+1 ~ MaZa)+ZaT ^ Za \ (28.43) 

Ca J 

one time step later. Likewise, linearizing the adjoint Fokker-Planck evolution 
operator (28.37) around the x a trajectory point yields: 



PaiZa) = 



— [dz a+ i]e ~ ha (28.44) 

a+l J 
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Completing the squares, integrating and substituting (28.41), respectively (28.42) 
we obtain the formula for covariance matrix evolution forward in time, 

Q a+ \ = M a Q a M a T + A a . (28.45) 

In the adjoint case, the evolution of the Q is given by 

M a Q a Ml = Q a+l + A a . (28.46) 

The two covariance matrices differ, as the adjoint evolution Q a is computed by 
going backwards along the trajectory. These covariance evolution rules are the 
basis of all that follows. 

Think of the initial covariance matrix (28.41) as an error matrix describing 
the precision of the initial state, a cigar-shaped probability distribution p a {z a )- In 
one time step this density is deterministically advected and deformed into density 
with covariance MQM T , and then the noise A is added: the two kinds of inde- 
pendent uncertainties add up as sums of squares, hence the covariance evolution 
law (28.45), resulting in the Gaussian ellipsoid whose widths and orientation are 
given by the singular values and singular vectors (6.13) of the covariance matrix. 
After n time steps, the variance Q a is built up from the deterministically propa- 
gated M n a Q a - n M n J initial distribution, and the sum of noise kicks at intervening 
times, M*A a _fcM* T , also propagated deterministically. 

The pleasant surprise is that the evaluation of this noise requires no Fokker- 
Planck PDE formalism. The width of a Gaussian packet centered on a trajectory 
is fully specified by a deterministic computation that is already a pre-computed 
byproduct of the periodic orbit computations; the deterministic orbit and its linear 
stability. We have attached label V to A a - A{x a ) in (28.45) to account for 
the noise distributions that are inhomogeneous, state space dependent, but time 
independent multiplicative noise. 



28.6 Weak noise: Hamiltonian formulation 

All imperfection is easier to tolerate if served up in small 
doses. 

— Wislawa Szymborska 

(G. Vattay and P. Cvitanovic) 

In the spirit of the WKB approximation (to be fully developed in chapter 32), we 
shall now study the evolution of the probability distribution by rewriting it as 

p{x,t) ^ e^ R(xA . (28.47) 
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The time evolution of R is given by 

d,R + vdR + (dR) 2 = Ddv + Dd 2 R . 

Consider now the weak noise limit and drop the terms proportional to D. The 
remaining equation 

d,R + H(x, dR) = 

is known as the Hamilton- Jacobi equation . The function R can be interpreted as 
the Hamilton's principal function, corresponding to the Hamiltonian 



H(x,p) = pv{x) + p 2 /2, 

with the Hamilton's equations of motion 

x - d p H = v + p 
p = -d x H = -A T p , 



(28.48) 



where A is the stability matrix (4.3) 

dXj 

The noise Lagrangian is then 

= = 5 [ *-* w l 2 - (2849) 

We have come the full circle - the Lagrangian is the exponent of our assumed 
Gaussian distribution (28.31) for noise £ 2 = [x - v(x)] 2 . What is the meaning 
of this Hamiltonian, Lagrangian? Consider two points an d x. Which noisy 
path is the most probable path that connects them in time t7 The probability of a 
given path f is given by the probability of the noise sequence £(?) which generates 
the path. This probability is proportional to the product of the noise probability 
functions (28.31) along the path, and the total probability for reaching x from xq 
in time t is given by the sum over all paths, or the stochastic path integral (Wiener 
integral) 



Pi v 1 

^ ^Z ex p(-4iX^ T ^ (T) )' (28 " 50) 
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where St/ = Tj - t,, and the normalization constant is 



— = lim 




The most probable path is the one maximizing the integral inside the exponential. 
If we express the noise (28.18) as 



By the standard arguments, for a given x, x' and t the probability is maximized by 
a solution of Hamilton's equations (28.48) that connects the two points xo — > x' 
in time t. The solution is a bit boring: x — v , p = , and lives in the initial, 
<i-dimensional state space, so not much is to be made of this surprising appearance 
of Hamiltonians. 



Resume 

When a deterministic trajectory is smeared out under the influence of Gaussian 
noise of strength D, the deterministic dynamics is recovered in the weak noise 
limit D — > 0. The effect of the noise can be taken into account by adding noise 
corrections to the classical trace formula. 

Symplectic structure emerges here not as a deep principle of mechanics, but 
an artifact of the leading approximation to quantum/noisy dynamics, not respected 
by higher order corrections. The same is true of semiclassical quantum dynamics; 
higher corrections do not respect canonical invariance. 



Remark 28.1 A brief history of noise. The theory of stochastic processes is a vast 
subject, starting with the Laplace 1810 memoir [28.42], spanning over centuries, and over 
disciplines ranging from pure mathematics to impure finance. The presentation given 
here is based on the Cvitanovic and Lippolis 2012 Maribor lectures [28.1]. The mate- 
rial reviewed is standard [28.2, 28.3, 28.43], but needed in order to set the notation for 
what is new here, the role that local Fokker-Planck operators play in defining stochastic 
neighborhoods of periodic orbits. We enjoyed reading van Kampen classic [28.2], espe- 
cially his railings against those who blunder carelessly into nonlinear landscapes. Having 



#f) - m - v{x{t)) , 



the probability is maximized by the variational principle 




Commentary 
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committed this careless chapter to print, we shall no doubt be cast to a special place on 
the long list of van Kampen's sinners (and not for the first time, either). A more special- 
ized monograph like Risken's [28.3] will do just as well. Schulman's monograph [28.12] 
contains a very readable summary of Kac's [28.13] exposition of Wiener's integral over 
stochastic paths. The standard Langevin equation [28.40] is a stochastic equation for a 
Brownian particle, in which one replaces the Newton's equation for force by two counter- 
balancing forces: random accelerations £(f) which tend to smear out a particle trajectory, 
and a damping term which drives the velocity to zero. In this context D is Einstein dif- 
fusion constant, and (28.11) is the Einstein diffusion formula [28.41]. Here we denote 
by 'Langevin equation' a more general family of stochastic differential equations (28.18) 
with additive or multiplicative [28.47, 28.48] weak noise. Noisy discrete time dynamical 
systems are discussed in refs. [28.60, 28.61, 28.62]. 

In probabilist literature [28.58] the differential operator -V ■ (v(x)p(x, t)) + D V 2 p(x, f) 
is called 'Fokker- Planck operator;' here we reserve the term 'Fokker-Planck evolution 
operator' for the finite time, 'Green's function' integral operator (28.30), i.e., the stochas- 
tic path (Wiener) integral [28.53, 28.54, 28.3] for a noisy flow, with the associated contin- 
uous time Fokker-Planck [28.2, 28.3, 28.55] (or forward Kolmogorov) equation (28.24). 

The cost function (28.31) appears to have been first introduced by Wiener as the ex- 
act solution for a purely diffusive Wiener-Levy process in one dimension, see (28.16). 
Onsager and Machlup [28.19, 28.24] use it in their variational principle to study ther- 
modynamic fluctuations in a neighborhood of single, linearly attractive equilibrium point 
(i.e., without any dynamics). It plays important role in the optimal control theory [28.63, 
28.64]. Gaussians are often rediscovered, so Onsager-Machlup seminal paper, which 
studies the same attractive linear fixed point, is in literature often credited for being the 
first to introduce a variational method -the "principle of least dissipation"- based on the 
Lagrangian of form (28.49). They, in turn, credit Rayleigh [28.20] with introducing the 
least dissipation principle in hydrodynamics. Onsager-Machlup paper deals only with a 
finite set of linearly damped thermodynamic variables, and not with a nonlinear flow or 
unstable periodic orbits. 

Gaspard [28.23] derives a trace formula for the Fokker-Planck equation associated 
with ltd stochastic differential equations describing noisy time-continuous nonlinear dy- 
namical systems. In the weak-noise limit, the trace formula provides estimations of the 
eigenvalues of the Fokker-Planck operator on the basis of the Pollicott-Ruelle resonances 
of the noiseless deterministic system, which is assumed to be non-bifurcating. At first 
order in the noise amplitude, the effect of noise on a periodic orbit is given in terms 
of the period and the derivative of the period with respect to the pseudo-energy of the 
Onsager-Machlup-Freidlin-Wentzell scheme [28.24]. The dynamical 'action' Lagrangian 
in the exponent of (28.30), and the associated symplectic Hamiltonian were first writ- 
ten down in 1970's by Freidlin and Wentzell [28.24], whose formulation of the 'large 
deviation principle' was inspired by the Feynman quantum path integral [28.49]. Feyn- 
man, in turn, followed Dirac [28.50] who was the first to discover that in the short-time 
limit the quantum propagator (imaginary time, quantum sibling of the Wiener stochastic 
distribution (28.16)) is exact. Gaspard [28.23] thus refers to the 'pseudo-energy of the 
Onsager-Machlup-Freidlin- Wentzell scheme.' M. Roncadelli [28.51, 28.52] refers to the 
Fokker-Planck exponent in (28.30) as the 'Wiener-Onsager-Machlup Lagrangian,' con- 
structs weak noise saddle-point expansion and writes transport equations for the higher 
order coefficients. In our exposition the setting is more general: we study fluctuations 
over a state space-varying velocity field v(x). 
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Remark 28.2 Weak noise perturbation theory. DasBuch omits any discussion of 
the Martin-Siggia-Rose [28.69] type weak noise corrections. For an overview of possible 
ways for improvement of diagrammatic summation in noisy field theories, see Chaotic 
Field Theory: a Sketch [28.70]. The details are in the three papers on trace formu- 
las for stochastic evolution operators (see also ref. [28.51]): Weak noise perturbation 
theory [16.9], smooth conjugation method [16.10], and local matrix representation ap- 
proach [16.11]. Such corrections have not been evaluated before, probably because one 
is so unsure about nature of the noise itself that nth order correction is beyond the point. 
Doing continuous time flows requires the same kind of corrections, with diagrams stand- 
ing for integrals rather than sums, though no one ever tried weakly stochastic flows in 
continuous time. 

Remark 28.3 Covariance evolution. In quantum mechanics the linearized evolu- 
tion operator corresponding to the linearized Fokker-Planck evolution operator (28.40) is 
known as the Van Vleck propagator, the basic block in the semi-classical periodic orbit 
quantization [30.2], see chapter 33. Q covariance matrix composition rule (28.45) or its 
continuous time version is called 'covariance evolution' (for example, in ref. [28.65]), but 
it goes all the way back to Lyapunov's 1892 thesis [28.66]. In the Kalman filter litera- 
ture [28.67, 28.68] it is called 'prediction'. 

Remark 28.4 Operator ordering. According to L. Arnold [28.43] review of the orig- 
inal literature, the derivations are much more delicate than what is presented here: the 
noise is colo red rather than Dirac delta function in time. He refers only to the linear case 
as the 'Langevin equation'. The 5t — » limit and the proper definition of i(r) are deli- 
cate issues [28.44, 28.45, 28.43, 28.46] of no import for the applications of stochasticity 
studied here: Ito and Stratanovich operator ordering issues arise in the order beyond the 
leading approximation considered here. 
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Exercises 

28.1. Who ordered ? Derive the Gaussian integral 

— — I dx e s = Va , a > . 
V27r J-co 

assuming only that you know to integrate the exponen- 
tial function e~ x . Hint, hint: x 2 is a radius-squared of 
something. 7r is related to the area or circumference of 
something. 

28.2. D-dimensional Gaussian integrals. Show that the 
Gaussian integral in D-dimensions is given by 

—L- f dV-i^'W = IdetMlUt^i) 

(27T) d '- J 

where M is a real positive definite [d x of] matrix, i.e., 
a matrix with strictly positive eigenvalues, x, J are D- 
dimensional vectors, and x T is the transpose of x. 

28.3. Convolution of Gaussians. Show that the Fourier 



transform of convolution 



[/ 



**](*) = j 



d d yf(x-y)g(y) 



of two Gaussians 

-I r T .-L-r 

f(x) = e , 
factorizes as 



[f*g]( 



(2n) d J 



--x T - — -x 

g(x) — e 1 A 2 



dk F{k)G{k)e ik x , (28.52) 



where 



xf(x)e~ ikx = |detAi| 1/z e 



U2 ik 



G(k) 



(2n) d J 



xg(x)e 



-it-; 



|detA 2 | 1/2 e^ 



Hence 



[/*£]« = 



1 



|det Aidet Ai| 1/2 j d d pe^ 

i.v T -(A 1 +A 2 r 1 -.v 



(2w) rf 

detAidetA 2 1/2 



det(Ai + A 2 ) 



exerNoise - 17apr2013 



ChaosBook.org version 14.4. 1 , Apr 212013 



REFERENCES 



590 



References 

[28.1] P. Cvitanovic and D. Lippolis. Knowing when to stop: How noise frees us 
from determinism. In M. Robnik and V. G. Romanovski, editors, Let's Face 
Chaos through Nonlinear Dynamics, pages 82-126, Melville, New York, 
2012. Am. Inst, of Phys. arXiv: 1206. 5506. 

[28.2] N. G. van Kampen, Stochastic Processes in Physics and Chemistry (North 
Holland, Amsterdam 1992). 

[28.3] H. Risken, The Fokker-Planck Equation: Methods of Solution and Appli- 
cations (Springer- Verlag, New York, 1996). 

[28.4] W. Dittrich and M. Reuter, Classical and Quantum Dynamics: From Clas- 
sical Paths to Path Integrals (Springer- Verlag, Berlin 1994). 

[28.5] E. M. Lifshitz and L. P. Pitaevskii, Physical Kinetics (Pergamon, London 
1981). 

[28.6] C. Itzykson and J.-M. Drouffe, Statistical field theory (Cambridge U. Press, 
1991). 

[28.7] V. Ambegaokar, Reasoning about luck; probability and its uses in physics 
(Cambridge Univ. Press, Cambridge 1996). 

[28.8] B. Sakita, Quantum theory of many variable systems and fields (World 
Scientific, Singapore 1985). 

[28.9] G. E. Uhlenbeck, G. W. Ford and E. W. Montroll, Lectures in Statistical 
Mechanics (Amer. Math. Soc, Providence R.I., 1963). 

[28.10] M. Kac, "Random walk and the theory of Brownian motion," (1946), 
reprinted in ref. [28.11]. 

[28. 11] N. Wax, ed., Selected Papers on Noise and Stochastic Processes (Dover, 
New York 1954). 

[28.12] L. S. Schulman, Techniques and Applications of Path Integration (Wiley- 
Interscience, New York 1981). 

[28.13] M. Kac, Probability and Related Topics in Physical Sciences (Wiley - 
Interscience, New York 1959). 

[28.14] E. Nelson, Quantum Fluctuations (Princeton Univ. Press 1985). 

[28. 15] H. Kunita, Stochastic Flows and Stochastic Differential Equations (Cam- 
bridge Univ. Press, Cambridge 1990). 

[28.16] H. Haken and G. Mayer-Kress, Z.f. Physik B 43, 185 (1981). 

[28.17] M. Roncadelli, Phys. Rev. E 52, 4661 (1995). 

[28.18] G. Ryskin, Phys. Rev. E 56, 5123 (1997). 

[28.19] L. Onsager and S. Machlup, Phys. Rev. 91, 1505, 1512 (1953). 



refsNoise - 24dec2012 



ChaosBook.org versionl4.4.1, Apr 21 2013 



References 



591 



[28.20] Lord Rayleigh, Phil. Mag. 26, 776 (1913). 

[28.21] G. E. Uhlenbeck and L. S. Ornstein, Phys. Rev. 36, 823 (1930). 

[28.22] M. Jacobsen, Bernoulli J. of Math. Stat, and Prob. 2, 111 (1997). 

[28.23] P. Gaspard, "Trace Formula for Noisy Flows," /. Stat. Phys. 106, 57 
(2002). 

[28.24] M. I. Freidlin and A. D. Wentzel, Random Perturbations of Dynamical 
Systems, (Springer, New York 1998). 

[28.25] E. Gozzi, M. Reuter and W. D. Thacker, Phys. Rev. D 40, 3363 (1989). 

[28.26] E. Gozzi and M. Reuter, Phys. Lett. 233B, 383 (1989); 238B, 451 (1990); 
240B, 137 (1990). 

[28.27] E. Gozzi, M. Reuter and W. D. Thacker, Phys. Rev. D 46, 757 (1992). 

[28.28] E. Gozzi, M. Reuter and W. D. Thacker, Chaos, Solitons and Fractals 2, 
441 (1992). 

[28.29] R. Benzi, G. Paladin, G. Parisi and A. Vulpiani, J. Phys. A 18, 2157 
(1985). 

[28.30] R. Graham, Europhys. Lett. 5, 101 (1988). 

[28.31] R. Graham and T. Tel, Phys. Rev. A 35, 1382 (1987). 

[28.32] L. Arnold and V. Wihstutz, Lyapunov exponents, Lecture Notes in Math. 
1186 (Springer- Verlag, New York 1986). 

[28.33] M. V. Feigelman and A. M. Tsvelik, Sov. Phys. JETP 56, 823 (1982). 

[28.34] G. Parisi and N. Sourlas, Nucl. Phys. B206, 321 (1982). 

[28.35] F. Langouche et al., Functional integration and semiclassical expansion 
(Reidel, Dordrecht 1982). 

[28.36] O. Cepas and J. Kurchan, "Canonically invariant formulation of Langevin 
and Fokker-Planck equations," European Phys. J. B 2, 221 (1998); 
arXiv : cond-mat/9706296. 

[28.37] S. Tanase-Nicola and J. Kurchan, "Statistical-mechanical formulation of 
Lyapunov exponents;" arXiv: cond-mat/021038®. 

[28.38] M. Abramowitz and I. A. Stegun, editors. Handbook of Mathematical 
Functions. Dover, New York, 1964. 

[28.39] H. Takada, Y. Kitaoka, and Y. Shimizua. Mathematical index and model 
in stabirometry. Forma, 16:17-46, 2001. 

[28.40] S. Chandrasekhar. Stochastic problems in physics and astronomy. Rev. 
Mod. Phys., 15:1, 1943. 



refsNoise - 24dec2012 



ChaosBook.org versionl4.4.1, Apr 21 2013 



References 



592 



[28.41] A. Einstein. Uber die von der molekularkinetischen Theorie der Warme 
geforderte Bewegung von in ruhenden Fliissigkeiten suspendierten Teilchen. 
Ann. Physik, 17:549, 1905. 

[28.42] P. S. Laplace. Memoire sur les integrates defmies et leur application aux 
probabilites, et specialement a la recherche du milieu qu'il faut choisir entre 
les resultats des observations. Mem. Acad. Sci. (I), XI, Section V, pages 
375-387, 1810. 

[28.43] L.Arnold. Stochastic Differential Equations: Theory and Applications. 
Wiley, New York, 1974. 

[28.44] K. Ito, Stochastic integral, Proc. Imp. Acad. Tokyo 20 (1944) 519-524. 

[28.45] R. L. Stratonovich, Conditional Markov Processes and Their Application 
to the Theory of Control, Elsevier, New York, 1968. 

[28.46] R. F. Fox, Gaussian stochastic processes in physics, Phys. Rep. 48 (1978) 
179-283. 

[28.47] C. Kuehn. Deterministic continutation of stochastic metastable equilibria 
via Lyapunov equations and ellipsoids, 2011. arXiv: 1106. 3479. 

[28.48] J. L. Doob. The Brownian movement and stochastic equations. Ann. 
Math., 43:351-369, 1942. 

[28.49] R. Feynman, A. Hibbs, Quantum Mechanics and Path Integrals, 
McGraw-Hill Book Co., New York, 1965. 

[28.50] P. A. M. Dirac, The Lagrangian in quantum mechanics, Phys. Z. Sowje- 
tunion 3 (1933) 64-72. 

[28.51] M. Roncadelli, Small-fluctuation expansion of the transition probability 
for a diffusion process, Phys. Rev. E 52 (1995) 4661. 

[28.52] A. Defendi, M. Roncadelli, Semiclassical approximation as a small-noise 
expansion, J. Phys. A 28 (1995) L515-L520. 

[28.53] B. Shraiman, C. E. Wayne, and P. C. Martin. Scaling theory for noisy 
period-doubling transitions to chaos. Phys. Rev. Lett., 46:935-939, 1981. 

[28.54] P. C. Martin, E. D. Siggia, H. A. Rose, Statistical mechanics of classical 
systems, Phys. Rev. A 8 (1973) 423. 

[28.55] Bernt 0ksendal. Stochastic Differential Equations. Springer, New York, 
2003. 

[28.56] H. Dekker and N. G. Van Kampen. Eigenvalues of a diffusion process 
with a critical point. Phys. Lett. A, 73:374-376, 1979. 

[28.57] P. Gaspard, G. Nicolis, A. Provata, and S. Tasaki. Spectral signature of 
the pitchfork bifurcation: Liouville equation approach. Phys. Rev. E, 51:74, 
1995. 



refsNoise - 24dec2012 



ChaosBook.org versionl4.4.1, Apr 21 2013 



References 



593 



[28.58] N. Berglund and B. Gentz, Noise-Induced Phenomena in Slow-Fast Dy- 
namical Systems: A Sample-Paths Approach (Springer, Berlin 2005). 

[28.59] M. Stone and P. Goldbart, Mathematics for Physics: A Guided Tour for 
Graduate Students (Cambridge Univ. Press, Cambridge, 2009). 

[28.60] Y. Kifer, On small random perturbations of some smooth dynamical sys- 
tems, Math. USSR-Izv. 8 (1974) 1083-1107. 

[28.61] M. J. Feigenbaum, B. Hasslacher, Irrational decimations and path- 
integrals for external noise, Phys. Rev. Lett. 49 (1982) 605-609. 

[28.62] A. Boyarsky, On the significance of absolutely continuous invariant mea- 
sures, PhysicaD 11 (1984) 130-146. 

[28.63] L. S. Pontryagin, V. Boltyanskii, R. Gamkrelidge, E. Mishenko, Math- 
ematical Theory of Optimal Processes, Interscience Publishers, New York, 
1962. 

[28.64] R. Bellman, Dynamic Programming, Princeton Univ. Press, Princeton, 
1957. 

[28.65] M. K. Tippett and S. E. Cohn. Adjoints and low-rank covariance repre- 
sentation. Nonlinear Processes in Geophysics, 8:331-340, 2001. 

[28.66] A. M. Lyapunov. The general problem of the stability of motion. Int. J. 
Control, 55:531-534, 1992. 

[28.67] R. E. Kalman, A new approach to linear filtering and prediction problems, 
Trans. ASME - J. Basic Engineering 82 (1960) 35^15. 

[28.68] H. D. I. Abarbanel, D. R. Creveling, R. Farsian, M. Kostuk, Dynamical 
state and parameter estimation, SIAM J. Appl. Math. 8 (2009) 1341-1381. 

[28.69] P. C. Martin, E. D. Siggia, and H. A. Rose. Statistical mechanics of 
classical systems. Physical Review A, 8:423-437, 1973. 

[28.70] P. Cvitanovic, Chaotic field theory: a sketch, Physica A288, 61 (2000); 
arXiv : nlin . CD/0001034. 



rcfsNoisc - 24dec2012 



ChaosBook.org vcrsionl4.4.1, Apr 21 2013 



